Lossless Source Coding Lecture Notes & Examples

variously referred to as source coding, lossless coding, noiseless coding or entropy coding exactly and without error, the original message. Lossless coding implies that the message may be decoded back to the original sequence of symbols. The converse of lossless coding (“lossy” coding) implies some degree of approximation of the original message. lossless coding may augment lossy coding, eg VQ indexes necessary in cases where the source message must be received without corruption, eg textual messages. Lossy coding is often employed in situations where some finite quantization or approximation is tolerable for various reasons. Perceptual considerations usually render audio and video information amenable to lossy compression techniques. Invariably lossy coding techniques are able to extract a significantly higher compression rate in any given system. The information rate, or entropy of a source, is defined in terms of the probability of occurrence Pr(si) of source symbol si H = − ∑ si∈S Pr(si) log2 {Pr(si)} bits/symbol (1)